<html><head>
      <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
   <title>8.3.&nbsp; Monotonically Increasing Row Keys/Timeseries Data</title><link rel="stylesheet" href="css/freebsd_docbook.css" type="text/css"><meta name="generator" content="DocBook XSL-NS Stylesheets V1.75.2"><link rel="home" href="book.html" title="The Apache HBase Book"><link rel="up" href="schema.html" title="Chapter&nbsp;8.&nbsp;HBase and Schema Design"><link rel="prev" href="number.of.cfs.html" title="8.2.&nbsp; On the number of column families"><link rel="next" href="keysize.html" title="8.4.&nbsp;Try to minimize row and column sizes"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">8.3.&nbsp;
  Monotonically Increasing Row Keys/Timeseries Data
  </th></tr><tr><td width="20%" align="left"><a accesskey="p" href="number.of.cfs.html">Prev</a>&nbsp;</td><th width="60%" align="center">Chapter&nbsp;8.&nbsp;HBase and Schema Design</th><td width="20%" align="right">&nbsp;<a accesskey="n" href="keysize.html">Next</a></td></tr></table><hr></div><div class="section" title="8.3.&nbsp; Monotonically Increasing Row Keys/Timeseries Data"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a name="timeseries"></a>8.3.&nbsp;
  Monotonically Increasing Row Keys/Timeseries Data
  </h2></div></div></div><p>
      In the HBase chapter of Tom White's book Hadoop: The Definitive Guide (O'Reilly) there is a an optimization note on watching out for a phenomenon where an import process walks in lock-step with all clients in concert pounding one of the table's regions (and thus, a single node), then moving onto the next region, etc.  With monotonically increasing row-keys (i.e., using a timestamp), this will happen.  See this comic by IKai Lan on why monotically increasing row keys are problematic in BigTable-like datastores:
      <a class="link" href="http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-values-are-bad/" target="_top">monotonically increasing values are bad</a>.  The pile-up on a single region brought on
      by monoticially increasing keys can be mitigated by randomizing the input records to not be in sorted order, but in general its best to avoid using a timestamp or a sequence (e.g. 1, 2, 3) as the row-key. 
  </p><p>If you do need to upload time series data into HBase, you should
  study <a class="link" href="http://opentsdb.net/" target="_top">OpenTSDB</a> as a
  successful example.  It has a page describing the <a class="link" href=" http://opentsdb.net/schema.html" target="_top">schema</a> it uses in
  HBase.  The key format in OpenTSDB is effectively [metric_type][event_timestamp], which would appear at first glance to contradict the previous advice about not using a timestamp as the key.  However, the difference is that the timestamp is not in the <span class="emphasis"><em>lead</em></span> position of the key, and the design assumption is that there are dozens or hundreds (or more) of different metric types.  Thus, even with a continual stream of input data with a mix of metric types, the Puts are distributed across various points of regions in the table.
 </p></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="number.of.cfs.html">Prev</a>&nbsp;</td><td width="20%" align="center"><a accesskey="u" href="schema.html">Up</a></td><td width="40%" align="right">&nbsp;<a accesskey="n" href="keysize.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">8.2.&nbsp;
      On the number of column families
  &nbsp;</td><td width="20%" align="center"><a accesskey="h" href="book.html">Home</a></td><td width="40%" align="right" valign="top">&nbsp;8.4.&nbsp;Try to minimize row and column sizes</td></tr></table></div></body></html>